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Background / Context: 

While the adjusted cohort graduation rate of students in the United States has reached 80 
percent during the 2011-12 school year (Stetser & Stillwell, 2014), in the School District of 
Philadelphia (SDP) only 57% of its students graduate from high school in four years, which is 
typical of large urban school districts (Swanson, 2009). Moreover, less than 20% of SDP 
students graduate high school, enroll in college, and persist to the second year (The School 
District of Philadelphia Action Plan, 2013). School attendance correlates with academic 
achievement and is also among the strongest predictors of high school graduation (Balfanz & 
Byrnes, 2012; Byrnes & Reyna, 2012). Educators care greatly about the amount of time students 
spend in the classroom because absenteeism results in substantial loss of classroom learning 
time. Teachers and administrators estimate that some students may lose not just days, but weeks 
of learning time, due to tardiness and absences. 

One of SDP’s priorities is to improve student attendance as part of its Action Plan. The 
district created a task force dedicated to re-think the existing report card structure that is sent to 
guardians. Currently, student performance data (e.g., course grades and attendance) is presented 
to guardians in report cards in a way that is often convoluted and confusing. The task force 
intends to develop new formatting and, potentially, new content for the report cards. By making 
absenteeism information more meaningful, relevant, and contextualized, guardians might be 
enabled to take on a more active role in improving their student’s attendance and academic 
perfonnance. 

The Student Social Support R&D Lab at Harvard (S3) and the Office of Research and 
Evaluation (ORE) at the School District of Philadelphia (SDP) collaborated on the “SDP 
Attendance Project”, a research project in the fonn of a randomized control trial (RCT) to study 
strategies to improve student attendance in SDP. The project developed a low-cost, easy-to- 
implement innovation that could be widely used to reduce student absences. Overall, the project 
generated between 15,000 and 20,000 increased days of attendance (fewer days of absence) 
among some of the district’s most at-risk students. This is valued at between $1,100,000 and 
$1,400,000. 

Successful execution of this project required a close collaboration between the S3 Lab 
team and SDP departments, including ORE, throughout all steps of the project. 

Purpose: 

Jointly reflecting on our RCT collaboration, we have identified characteristics of the 
district and the research team that made this a successful project. We hope to share these 
reflections in this session so that other research collaborations can be more effective. 

Settings: 

Description of SDP: SDP is the eighth largest school system in the country, serving over 
200,000 preK-12 students. The District enrolls a diverse population of students: 9.4% of students 
are classified as ELLs, and 13.8% as having disabilities; 52% are African American, 19% are 
Hispanic/Latino, 14% are Caucasian/Euro-Americans, 8% are Asian, and 4% identify as 
multiracial or other (SDP, 2014a, 2014b). SDP seeks to build on it successes and address its 
challenges to better serve its students through innovative avenues and partnerships. The Office of 
Research and Evaluation is in charge of research and program evaluations at SDP. Their focus is 
to develop data-driven information to be used by SDP and the City of Philadelphia to produce 
the knowledge needed to improve programs and services for the students and families of SDP. 
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Description of S3: The Student Social Support R&D Lab (S3 R&D Lab) is led by Todd 
Rogers, Associate Professor of Public Policy at the Harvard Kennedy School. The team uses data 
and behavioral science to develop and prove easy-to-implement, high return-on-investment 
interventions that mobilize and empower students’ social support systems to improve 
achievement. 

Research Study Design: 

The project’s objective is to inform and motivate parents/guardians to reduce student 
absences through multiple mail-based communications during the school year. The study 
involves 30,000 grade 1-12 students (and their guardians) enrolled in public elementary, middle, 
and high schools in the 2014-15 school year in SDP (about 200 schools total). Participating 
households were randomly assigned to one of four conditions: (1) a Control group that receives 
no additional outreach as part of this experiment, (2) an Encourage treatment group that receives 
mail stressing the importance of attendance, parental efficacy, and absence-reduction as part of 
parental role, (3) an Encourage + Self group that receives mail that has the same content as the 
Encourage mail but also states the number of days of school the guardians’ student missed, and 
(4) an Encourage + Self + Norms group that receives mail that contains all the same information 
as the Encourage + Self mail but also compares the student’s attendance record to that of the 
typical student in their school and grade. Those assigned to the three treatment arms received 5 
mailings over the course of the 2014-2015 school year. Mailing 1 showed absence records from 
the 2013-14 school year and mailing rounds 2, 3, 4, and 5 reflect up-to-date absence records 
from the 2014-2015 school year. 

The project addresses the following research questions (analysis plan pre-registration: X): 

1. Does contacting guardians and encouraging them to improve their students’ attendance 
reduce absences? 

2. Does communicating to guardians the total number of days their student missed reduce 
absences? That is, are parents miscalibrated on their beliefs about how many days their 
students have been absent, and does this miscalibration matter? 

3. Does communicating to guardians the total number of days their student missed as 
compared to the absences of a typical student reduce absences? That is, are parents 
miscalibrated on their beliefs about how their students’ number of absences compare to 
their students’ classmates’ absences, and does this miscalibration matter? 

There is one more exploratory ex ante research question: 

4. Do these interventions also impact the attendance of other students in the household not 
explicitly mentioned in the mailings (e.g., siblings)? 


Data Collection and Analysis: 

The project incorporated various data sets exported directly from the administrative 
records of SDP. The data sets included student demographics and enrollment data, guardian 
contact information, and attendance data. The research team also conducted a live phone survey 
of guardians in our experimental universes after Mailing 5 (between 6/20/2015-6/25/2015). The 
survey has two primary purposes: (1) internal validity - a set of questions address whether the 
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guardians received, read, and understood the mail and (2) assessing impact on parental beliefs - 
how did the mail pieces impact parental beliefs about the importance of attendance and their role 
in ensuring their students get to school? 

Data analyses were conducted using OLS regressions to generate treatment effect 
magnitudes and one-sided Fisher Randomization Tests to evaluate statistical significance (i.e. p- 
values). The primary outcome of interest was the mailings’ impact on the number of recorded 
absences between the day immediately after mailing #1 was sent out and the end of the school 
year. Households with two students who were both eligible for inclusion in the experiment were 
also analyzed to evaluate the possibility of spillover effects. 

Research Project - Results: 

We see strong evidence that the attendance mail reduced absences. We find that the 
Encourage component reduced absences by 0.60 days as compared to Control (p<0.001). The 
5e//~component reduced absences by an additional 0.45 days as compared to Encourage 
(p<0.001). There was no significant reduction in absences caused by the addition of the Norms 
component above the effect of Encourage + Self, however (p=0.24). 

Additionally, there were indications that these mailings had spillover effects on other 
students in the household. Namely, while in two student households, the Self and Norms 
mailings reduced the focal student’s absences by approximately a day (p<.001), the non-focal 
student’s absences were also reduced by .42 days (p=.018). 

A phone survey of guardians provided strong evidence of internal validity for the RCT. 
The mail successfully reached guardians, guardians read and comprehended it, and it changed 
the beliefs for the relevant conditions regarding the number of days their students had been 
absent and how those absences compared to their classmates. 

Overall, this project generated between 15,000 and 20,000 increased days of attendance 
(fewer days of absence) among some of the district’s most at-risk students. This is valued at 
between $1,100,000 and $1,400,000. 

Findings: The Partnership 

Our collaboration has resulted in sustained and increased dialogue about strategies to 
improve attendance across the district. We have been able to share details about the project and 
partnership at SDP events, such as the annual SDP Research, Policy, and Practice Conferences. 
Jointly reflecting on the project, we have identified characteristics of the district and the research 
team that have made this collaboration successful and possible. We hope to share these 
reflections in this session so that other research collaborations can be more effective. 

Project Design and Set-Up . 

From the perspective of SDP 

• The project aimed to improve student attendance, a priority for SDP. 

• The project was also straightforward for SDP to implement. It required regular 
monitoring and data sharing, but implementation was handled by the S3 Lab at Harvard. 

• The S3 Lab was careful to comply with all relevant regulations involving FERPA, human 
subjects research, and data security. 

• There were adequate resources. The S3 Lab raised funds to pay for the project, and offset 
some of the districts costs for monitoring and data sharing. 
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• The SDP team carefully monitored the project, so SDP was always informed. 

• The SDP team maintained district buy-in by keeping district leadership involved and 
infonned from the very beginning of the project. 

• The SDP team had capable, informed, and motivated people who were the S3 Lab’s 
regular points of contact for immediate logistical and implementation questions. 

• The SDP team linked the S3 Lab researchers with the district attendance and truancy staff 
to ensure they were aware of the project and had the opportunity to share feedback. 

• The research teams were transparent to parents and school leaders, and were extremely 
responsive to anyone with questions. 

From the perspective of the S3 Lab. 

• Data security, FERPA compliance, and Human Subjects Research compliance were 
paramount concerns. This helped earn the trust of SDP’s and Harvard’s relevant leaders 
while also maintaining the rights of research participants. 

• The S3 Lab maintained regular communications (nearly weekly) with relevant Harvard 
and SDP compliance officers to ensure that rights were protected. 

• A similar approach was taken with mail printing vendors - the S3 team chose vendors 
focused on data-security while also be responsive to modifications. These characteristics 
were crucial to ensure mailings remained on track and that the vendors could work within 
the parameters of the project. 

• All S3 Lab team members working on the project or handling data had the appropriate 
certifications, i.e. CITI or NIH Certifications. 

• The S3 Lab secured reviews and approvals for the project (IRB, data agreements, and 
vendor agreements). 

• The S3 Lab used secure data systems to safely store, analyze, and transfer student data. 

• The S3 Lab team was able to scale up staff support for the project during periods of high 
demand, such as when opt-outs came in after the initial infonned consent mailing. 

• The S3 Lab team was responsive and willing to adjust the implementation of the RCT 
based on SDP or parent feedback, while maintaining the fidelity of the RCT. 

• Each mailing allowed guardians to contact the research team with any questions or 
concerns, and to opt out. All people who fielded phone calls participated in a training 
process and used a pre-detennined script for replying to emails and calls. 


Conclusions 

This collaboration was a success for both the district and the research team. It generated 
15,000 to 20,000 net days of increased attendance by reducing absences among some of the most 
at-risk students in the district. It also generated deep insights into the influence of parental 
beliefs on student absences, while also expanding more basic understanding of human 
motivation. We believe other researchers and practitioners will benefit from the best practices 
we have learned from this two year collaboration. 
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